The RBSE Spider - Balancing Effective Search Against Web Load
نویسنده
چکیده
The design of a Web spider entails many things, including a concern for reasonable behavior, as well as more technical concerns. The RBSE Spider is a mechanism for exploring World Wide Web structure and indexing useful material thereby discovered. We relate our experience in constructing and operating this spider.
منابع مشابه
The RBSE Spider - Balancing Effective Search Against Web Load1
The design of a Web spider entails many things, including a concern for reasonable behavior, as well as more technical concerns. The RBSE Spider is a mechanism for exploring World Wide Web structure and indexing useful material thereby discovered. We relate our experience in constructing and operating this spider.
متن کاملLoad Balancing Approaches for Web Servers: A Survey of Recent Trends
Numerous works has been done for load balancing of web servers in grid environment. Reason behinds popularity of grid environment is to allow accessing distributed resources which are located at remote locations. For effective utilization, load must be balanced among all resources. Importance of load balancing is discussed by distinguishing the system between without load balancing and with loa...
متن کاملData Partition and Job Scheduling Technique for Balancing Load in Cluster of Web Spiders
Search engines primary rely on web spiders to collect large amount of data for indexing and analysis. Data collection can be performed by several agents of web spiders running in parallel or distributed manner over a cluster of workstations. This parallelization is often necessary in order to cope with a large number of pages in a reasonable amount of time. However, while keeping a good paralle...
متن کاملAn Effective Distributed Search Technique for Unstructured Peer-to-Peer Networks
This paper presents an efficient distributed search technique for unstructured P2P networks. The scheme employs Q-learning, power peers, specialized peers, load balancing, and mobile agents. Walkers are selected from neighbors as well as power peers. Each search process updates Q-values of nodes in different Q-tables. The mobile agent based load balancing scheme assists to route the queries to ...
متن کاملMulticriterion Evolutionary Algorithm for Workload Balancing of the Web Bank Servers
Load balancing of the Internet bank can be implemented by reduction of the workload of the bottleneck Web server. Load balancing improves both a performance of the computer system and the reliability of a server network. An evolutionary algorithm improved by introducing a tabu search procedure is discussed for solving multi-criteria optimization problem of finding a set of Pareto-suboptimal tas...
متن کامل